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4-Introduction 


1.1 Objective 


The current document discusses approach for continuous 
validation in the DevOps context for < application name > 
application. This is a living document that may be refined as the 
project progresses. The Project stakeholders - QA Manager, Test 
team lead, Project Manager and Develop manager would review 
and approve the final version of the document. 


This continuous validation test strategy document will target the 
highlighted area within the entire DevOps lifecycle. 
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1.2 Continuous Validation 


Question < Reason for adopting continuous validation for the 
current application and intended benefits to be derived post 
implementation >? 

Ans:- Adopting continuous validation for your current application can 
bring several significant benefits. Here are some key reasons and the 


intended benefits post-implementation: 
Reasons for Adopting Continuous Validation 
1. Improved Code Quality: 

o Continuous validation ensures that code is constantly tested for 
functionality, security, and performance, leading to higher quality 
code’. 

2. Early Detection of Issues: 

o By validating code continuously, potential issues are identified and 

resolved early in the development cycle, reducing the risk of major 


defects!. 
3. Enhanced Compliance: 
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o Continuous validation helps in maintaining compliance with 
industry standards and regulations by regularly checking code 
against these requirements’. 

4. Increased Efficiency: 

o Automating the validation process reduces the need for manual 

checks, allowing developers to focus on more value-added tasks’. 
o. Better Alignment with Business Goals: 

o Continuous validation ensures that the development efforts are 
aligned with business objectives by regularly reviewing and 
validating the strategy’. 

Intended Benefits Post-Implementation 
1. Reduced Deployment Risks: 

o Continuous validation minimizes the risk of deploying faulty code, 

leading to more stable and reliable releases’. 
2. Faster Time-to-Market: 

o With automated validation, the development process becomes more 

efficient, enabling quicker releases and updates’. 
3. Improved Customer Satisfaction: 

o Higher quality and more reliable applications result in better user 

experiences and increased customer satisfaction’. 
4. Cost Savings: 

o Early detection and resolution of issues reduce the cost associated 

with fixing defects later in the development cycle’. 
o. Scalability: 

o Continuous validation supports scalable development practices, 
allowing the application to grow and adapt to new challenges and 
requirements’. 

Implementing continuous validation can significantly enhance the overall 
development process, leading to more robust and reliable applications. 


Is there a specific aspect of continuous validation you’re particularly 
interested in? 


Question < Provide the details based on the current Gap analysis 
for continuous validation and high level overview of how it would 
be approached > 

Answer:- Gap Analysis for Continuous Validation 

Current State: 


1. Manual Testing: 
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o Testing is performed manually, leading to inconsistencies 
and potential human errors. 
2. Delayed Issue Detection: 
o Issues are often detected late in the development cycle 
increasing the cost and effort required to fix them. 
3. Lack of Automation: 
o Limited or no automation in the validation process, resultin 
in slower feedback loops. 
4, Inconsistent Compliance: 
o Difficulty in maintaining consistent compliance with 


industry standards and regulations. 
35. Resource Intensive: 
o Manual validation is resource-intensive, requiring significant 
time and effort from the development team. 
Desired State: 
1, Automated Testing: 
o Implementing automated tests to ensure consistent and 
reliable validation. 
2. Early Issue Detection: 
o Continuous validation to detect and resolve issues early in 
the development cycle. 
3. Enhanced Automation: 
o Leveraging automation tools to streamline the validation 
process and provide faster feedback. 
4, Consistent Compliance: 
o Regular automated checks to ensure compliance with 
industry standards and regulations. 
3. Efficient Resource Utilization: 
o Reducing the manual effort required for validation, allowin 
the team to focus on more critical tasks. 
High-Level Approach to Implement Continuous Validation 
1, Assessment and Planning: 
o Conduct a thorough assessment of the current validation 
processes. 
o Identify key areas for improvement and define the scope of 
continuous Validation. 
o Develop a detailed implementation plan, including timelines 
and resource allocation. 
2. Tool Selection: 
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o Evaluate and select appropriate automation tools for 
continuous Validation (e.g., Jenkins, Selenium, JUnit). 
o Ensure the chosen tools integrate well with the existing 
development environment. 
3. Test Automation: 
o Develop automated test scripts for various validation 


scenarios (unit tests, integration tests, performance tests). 


o Implement continuous integration (CI) pipelines to run 


automated tests on every code commit. 
4, Compliance and Standards: 

o Define and implement automated checks for compliance with 
industry standards and regulations. 

o Regularly update the validation criteria to align with 
evolving standards. 

3. Monitoring and Reporting: 

o Set up monitoring tools to track the results of continuous 
validation. 

o Generate regular reports to provide insights into the 
validation process and identify areas for further 
improvement. 

6. Training and Support: 

o Provide training to the development team on the new 
validation processes and tools. 

o Establish a support system to address any issues or 
Challenges encountered during implementation. 

7. Continuous Improvement: 

o Regularly review and refine the continuous validation 
process to ensure it remains effective and efficient. 

o Incorporate feedback from the development team to make 
necessary adjustments. 

Implementing continuous validation will help you achieve higher 
code quality, early issue detection, and more efficient resource 
utilization, ultimately leading to a more robust and reliable 
application. 

Would you like more details on any specific step or tool 


mentioned? 
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1.3 Application Overview 


Question < Provide Application overview > 


Answer:- Application Overview for DevOps of a New Project 
Purpose 


The purpose of this overview is to outline the key aspects of a new 
project from a DevOps perspective, ensuring efficient 
development, deployment, and maintenance processes. 


Key Features 


1. Admin Interface: 


o User Management: Admins can manage user roles and 
permissions. 


o Content Management: Admins can add, edit, and delete 
content. 


2. User Interface: 


o Responsive Design: Ensures the application is accessible on 
various devices. 


o User-Friendly Navigation: Intuitive navigation for a seamless 
user experience. 


3. Data Persistence: 


o Database Integration: Use of a robust database system for 
data storage and retrieval. 


o Backup and Recovery: Regular backups and recovery plans 


to ensure data integrity. 
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Technical Overview 


1. Frontend: 


o Frameworks: Use modern frameworks like React or Angular 
for building the user interface. 


o Styling: Use CSS frameworks like Bootstrap or Tailwind CSS 
for responsive design. 


2. Backend: 


o APTI Development: Develop RESTful APIs using frameworks 
like Spring Boot or Express.js. 


o Database: Use relational databases like PostgreSQL or 
MySQL, or NoSQL databases like MongoDB. 


3. Automation and CI/CD: 


o Continuous Integration (CI): Automated builds and tests 
using tools like Jenkins or GitHub Actions. 


o Continuous Deployment (CD): Automated deployment 


processes using Docker and Kubernetes. 
DevOps Considerations 
1. Infrastructure as Code (IaC): 


o Use tools like Terraform or Ansible to manage infrastructure 
setup and configuration. 


2.CI/CD Pipeline: 
o Version Control: Use Git for source code management. 
o Build Automation: Use Jenkins or GitHub Actions for 


automated builds. 


o Testing: Implement automated testing (unit, integration, 
and performance tests) using tools like JUnit and Selenium. 
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o Deployment: Use Docker for containerization and 
Kubernetes for orchestration. 


3. Monitoring and Logging: 


o Implement monitoring tools like Prometheus and Grafana to 
track application performance. 


o Use logging tools like ELK Stack (Elasticsearch, Logstash, 
Kibana) for centralized logging and analysis. 


4. Security: 


o Implement security best practices, including automated 
security scans and vulnerability assessments. 


o Use tools like OWASP ZAP for security testing. 
5. Scalability and Reliability: 


o Design the application to be scalable, using load balancers 
and auto-scaling groups. 


o Ensure high availability with redundant systems and failover 
mechanisms. 


Benefits for DevOps 


« Automated Processes: Reduced manual intervention with 


automated builds, tests, and deployments. 


« Early Issue Detection: Continuous validation and testing to catch 
issues early in the development cycle. 


- Efficient Resource Utilization: Optimized use of resources 
through automation and monitoring. 


« Scalability: Easily scalable infrastructure to handle growing 
demands. 


« Enhanced Security: Regular security checks and automated 


vulnerability assessments. 
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Implementing these DevOps practices will ensure a robust, 
scalable, and secure application, providing a seamless experience 


for both admins and users. 


Would you like more details on any specific DevOps tool or 
practice? 
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2. Continuous Validation Test Strategy 


2.1 


2.2 


Testing objective 


To test and prove that the requirements, features and code 
are fit to move into the next level. 


System, Acceptance testing will provide the information as 
to whether the features are ready for the production. 


Mitigate the product risks, highlight issues and new risks. 
Finding defects sooner through continuous testing 
Faster feedback through automated testing 


Test Approach 


To successfully test and deliver the product in continuous 
validation environment. 


Test as early as possible - 
Test as often as possible - 
= Continuously validate with every deployment or code 
upgrade 
= Adapt and Evolve with each delivery and test cycle 
Test maximum & Prioritize 
Test most important features first 


The features/requirements that are planned to be delivered 
for iteration will be reviewed for product risk and an 
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appropriate testing approach will be developed for each 
iteration. 
e Automation Regression, Build Verification testing 


= 100% Automated Execution 
= Execute automation scripts for every deployment 


Test cases development will base on the documentation 
(Requirements, Story boards) available. The key difference is test 
Case prioritization, as discussed above it be carried out after 
every scrum meeting. 


2.3 Test Excellence - Adhering to testing standard 


As a part of the mature testing delivery practices, the following 
standards will be followed during the continuous validation. 


2.3.1 Test Data Management 


< Provide the automated test data set up strategy for the 
current application, identify the TD needs - ex: whether mockup 
is required, how frequently refresh is required, what are the test 
data manipulation needs - ex: add, modify or delete etc. > 


Answer:-Automated Test Data Setup Strategy for New DevOps 
Project 
Test Data Needs 


1. Mockup Data: 
o Requirement: Yes, mockup data is essential to simulate real- 
world scenarios for testing. 
o Types: User data, transaction records, configuration settings, 
and other relevant data specific to the application. 
2. Data Refresh Frequency: 


o Frequency: Test data should be refreshed before each major 
testing cycle (e.g., before integration testing, system 


testing). 
o Automated Refresh: Implement automated scripts to refresh 


test data at scheduled intervals or on-demand. 


3. Test Data Manipulation Needs: 
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o Add: Ability to add new test data entries to test new features 
or scenarios. 
o Modify: Ability to modify existing test data to test different 
conditions and edge cases. 
o Delete: Ability to delete test data to ensure proper handling 
of data removal and cleanup. 
Automated Test Data Setup Strategy 
1. Data Generation: 
o Automated Scripts: Develop scripts to generate mockup 
data, ensuring it covers a wide range of scenarios. 
o Data Variability: Ensure the generated data includes various 
edge cases and typical use Cases. 
2. Data Storage: 
o Database: Use a separate test database to store the 
generated test data. 
o File System: Store files and other non-database data in a 
designated test directory. 
3. Data Refresh: 
o Scheduled Jobs: Set up scheduled jobs to refresh test data at 
specified intervals. 
o On-Demand Refresh: Provide the ability to trigger data 
refresh manually when needed. 


4. Data Manipulation: 
APIs for Test Data Management: Develop APIs to add 


modify, and delete test data entries. 

o User Interface: Provide a simple UI for testers to manage 
test data if needed. 

5. Integration with CI/CD Pipeline: 

o Pre-Test Setup: Include test data setup scripts in the CI/CD 
pipeline to ensure the test environment is prepared before 
running tests. 

o Post-Test Cleanup: Implement cleanup scripts to remove test 


data after testing is complete. 
6. Version Control: 
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o Test Data Scripts: Store test data generation and 
manipulation scripts in version control (e.g., Git) to track 
changes and ensure consistency. 

7. Monitoring and Logging: 

o Logs: Implement logging for test data setup and 
manipulation activities to track any issues. 

o Monitoring: Monitor the test data environment to ensure 
data integrity and availability. 

By implementing this automated test data setup strategy, you 
can ensure that your new DevOps project is thoroughly tested 
with realistic data, leading to higher quality and more reliable 
releases. 

Would you like more details on any specific aspect of this 
strategy? 


2.3.2 Test Management 


Question< Provide the test management tool details, standards for 
test cases design, automated test cases execution status update 
and defects flow etc for the current application > 


Answer:- Test Management Tool Details 


Recommended Test Management Tools 
1. PractiTest: 
o Features: End-to-end SaaS test management platform, 


integrates with various CI/CD tools, supports agile and 
waterfall methodologies. 


o Benefits: Centralized management of test Cases, real-time 
reporting, and seamless integration with bug tracking tools’. 


2. TestRail: 
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o Features: Comprehensive test Case management, 
customizable dashboards, and detailed reporting. 


o Benefits: Easy integration with CI/CD pipelines, supports 
manual and automated testing, and provides robust 
analytics’. 


3. Tricentis qTest: 


o Features: Supports agile and DevOps workflows, integrates 
with Jira, and provides real-time insights. 


o Benefits: Scalable for large teams, enhances collaboration, 
and ensures traceability’. 


Standards for Test Case Design 
1. Clarity and Conciseness: 
o Test cases should be clear, concise, and easy to understand. 
o Use simple language and avoid ambiguity. 
2. Traceability: 


o Each test case should be traceable to specific requirements 
or user stories. 


o Include references to requirement IDs or user story IDs. 
3. Reusability: 


o Design test cases to be reusable across different test 
scenarios. 


o Modularize test steps to avoid duplication. 
4, Coverage: 
o Ensure comprehensive coverage of all functional and non- 
functional requirements. 
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o Include positive, negative, and edge case scenarios. 
3. Maintainability: 
o Test cases should be easy to maintain and update. 


o Use consistent naming conventions and document any 
dependencies. 


Automated Test Cases Execution Status Update 
1. Integration with CI/CD Pipeline: 


o Integrate automated test cases with the CI/CD pipeline using 
tools like Jenkins or GitHub Actions. 


o Ensure test cases are executed automatically on code 
commits or scheduled intervals. 


2. Real-Time Reporting: 


o Use test management tools to provide real-time updates on 
test execution status. 


o Generate automated reports and dashboards to track test 
progress and results. 


3. Notifications: 
o Set up notifications for test execution status updates. 
o Notify relevant stakeholders of test results, failures, and any 


critical issues. 
Defects Flow 
1. Defect Reporting: 


o Report defects using the integrated bug tracking tool (e.g., 
Jira, Bugzilla). 
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o Include detailed information such as steps to reproduce, 
expected vs. actual results, and severity. 


2. Defect Triage: 


o Conduct regular defect triage meetings to prioritize and 
assign defects. 


o Ensure defects are categorized based on severity and impact. 
3. Defect Resolution: 
o Assign defects to the appropriate developers for resolution. 


o Track the progress of defect resolution and ensure timely 
fixes. 


4, Defect Verification: 


o Verify fixed defects through retesting. 


o Update the defect status to “Closed” once the fix is 
confirmed. 


3. Defect Metrics and Reporting: 


o Track defect metrics such as defect density, resolution time, 
and reopen rate. 


o Generate reports to analyze defect trends and identify areas 
for improvement. 


Implementing these standards and practices will help ensure 
efficient test management, accurate test execution status updates, 
and effective defect handling for your current application. 


Would you like more details on any specific tool or process? 


2.3.3 SOA and Service Virtualization 
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Question< Identify SOA testing needs for an API based 
application. Provide the details around why service virtualization 
and which all services have been virtualized, tools used for service 
virtualization etc. for the current application. > 


Answer:- SOA Testing Needs for an API-Based Application 
SOA Testing Needs 
1. Functional Testing: 


o Ensure that each service performs its intended function 
correctly. 


o Validate the integration between different services. 
2. Performance Testing: 


o Assess the performance of services under various load 
conditions. 


o Identify bottlenecks and optimize service performance. 
3. Security Testing: 
o Verify that services are secure and protected against 


vulnerabilities. 
o Ensure compliance with security standards and regulations. 
4. Interoperability Testing: 


o Ensure that services can interact seamlessly with other 
services and systems. 


o Validate data exchange and communication protocols. 
5. Regression Testing: 
o Ensure that new changes do not negatively impact existing 
functionality. 
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o Automate regression tests to maintain consistency. 
Why Service Virtualization? 
1. Dependency Management: 


o Service virtualization allows testing of services even when 
dependent components are unavailable or incomplete’. 


o It simulates the behavior of dependent services, enabling 
continuous testing without waiting for all components to be 
ready’. 


2. Cost Efficiency: 


o Reduces the need for setting up and maintaining complex 
test environments. 


o Saves costs associated with using third-party services for 
testing’. 


3. Parallel Development: 


o Enables parallel development and testing by providing 
virtual services that mimic real ones’. 


o Facilitates early detection of issues, reducing the overall 
development cycle time’. 


4. Consistent Testing Environment: 


o Provides a stable and consistent testing environment, 
reducing flakiness in test results’. 


o Ensures reliable and repeatable test outcomes. 
Services Virtualized 
1. Payment Gateway: 


o Simulates interactions with external payment processing 
services. 
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transactions. 


2. User Authentication: 


o Virtualizes authentication services to test login, logout, and 
session management. 


o Ensures security and functionality without relying on live 
authentication servers. 


3. Third-Party APIs: 


o Simulates responses from third-party APIs (e.g., weather 


services, social media integrations). 


o Enables testing of API interactions without depending on 
external services. 


4. Database Services: 


o Virtualizes database interactions to test CRUD operations. 


o Ensures data integrity and performance without using a live 
database. 


Tools Used for Service Virtualization 
1. Parasoft Virtualize: 
o Provides comprehensive service virtualization capabilities. 


o Supports a wide range of protocols and integrates with 
CI/CD pipelines’. 


2. WireMock: 


o An open-source tool for mocking HTTP services. 


o Easy to set up and use for simulating RESTful APIs?. 
3. Mountebank: 
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o An open-source tool that supports multiple protocols (HTTP, 
HTTPS, TCP). 


o Allows for complex service virtualization scenarios’. 
4. SmartBear ServiceV Pro: 


o Offers advanced service virtualization features. 


o Integrates with other SmartBear tools for end-to-end 
testing’. 


5. CA Service Virtualization (Broadcom): 


o Provides robust service virtualization for complex enterprise 
environments. 


o Supports a wide range of protocols and use cases’. 


Implementing service virtualization in your SOA testing strategy 
will enhance your ability to test services independently, reduce 
dependencies, and improve overall testing efficiency. 


Would you like more details on any specific tool or aspect of 


service virtualization? 


2.3.4 Test Automation 


Question< Provide the details around the framework used for the 
automation (BDD, TDD, Hybrid etc.), tools for automation (QTP, 
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Selenium, fitnesse, jBehave etc.), test cases have been automated 
etc. for the current application > 


Answer:- SOA Testing Needs for an API-Based Application 
SOA Testing Needs 
1. Functional Testing: 


o Ensure that each service performs its intended function 
correctly. 


o Validate the integration between different services. 
2. Performance Testing: 


o Assess the performance of services under various load 
conditions. 


o Identify bottlenecks and optimize service performance. 
3. Security Testing: 
o Verify that services are secure and protected against 


vulnerabilities. 


o Ensure compliance with security standards and regulations. 
4. Interoperability Testing: 


o Ensure that services can interact seamlessly with other 
services and systems. 


o Validate data exchange and communication protocols. 
5. Regression Testing: 


o Ensure that new changes do not negatively impact existing 
functionality. 


o Automate regression tests to maintain consistency. 


Why Service Virtualization? 
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1. Dependency Management: 


o Service virtualization allows testing of services even when 
dependent components are unavailable or incomplete’. 


o It simulates the behavior of dependent services, enabling 
continuous testing without waiting for all components to be 
ready’. 


2. Cost Efficiency: 


o Reduces the need for setting up and maintaining complex 
test environments. 


o Saves costs associated with using third-party services for 
testing’. 


3. Parallel Development: 


o Enables parallel development and testing by providing 
virtual services that mimic real ones’. 


o Facilitates early detection of issues, reducing the overall 
development cycle time’. 


4. Consistent Testing Environment: 


o Provides a stable and consistent testing environment, 
reducing flakiness in test results’. 


o Ensures reliable and repeatable test outcomes. 
Services Virtualized 
1. Payment Gateway: 


o Simulates interactions with external payment processing 
services. 


o Allows testing of various payment scenarios without actual 
transactions. 
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2. User Authentication: 


o Virtualizes authentication services to test login, logout, and 
session management. 


o Ensures security and functionality without relying on live 
authentication servers. 


3. Third-Party APIs: 


o Simulates responses from third-party APIs (e.g., weather 


services, social media integrations). 


o Enables testing of API interactions without depending on 
external services. 


4. Database Services: 


o Virtualizes database interactions to test CRUD operations. 


o Ensures data integrity and performance without using a live 
database. 


Tools Used for Service Virtualization 
1. Parasoft Virtualize: 
o Provides comprehensive service virtualization capabilities. 


o Supports a wide range of protocols and integrates with 
CI/CD pipelines’. 


2. WireMock: 
o An open-source tool for mocking HTTP services. 
o Easy to set up and use for simulating RESTful APIs?. 


3. Mountebank: 


o An open-source tool that supports multiple protocols (HTTP, 
HTTPS, TCP). 
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o Allows for complex service virtualization scenarios’. 


4. SmartBear ServiceV Pro: 


o Offers advanced service virtualization features. 


o Integrates with other SmartBear tools for end-to-end 
testing’. 


5. CA Service Virtualization (Broadcom): 


o Provides robust service virtualization for complex enterprise 
environments. 


o Supports a wide range of protocols and use cases’. 


Implementing service virtualization in your SOA testing strategy 
will enhance your ability to test services independently, reduce 
dependencies, and improve overall testing efficiency. 


Would you like more details on any specific tool or aspect of 
service virtualization? 


2.3.5 Continuous Improvement through Feedback 


Under the continuous improvement, any defect or deviation 
from the expected result will automatically be logged in the 
defect management tool specified for the delivery and 
notification to be sent to the defect owner and project manager. 


As a part of continuous improvement and feedback mechanism, 
there will also be required to conduct analysis on the defects 
found for the application and an improvement plan has to be 
prepared at end of each sprint or pre-decided interval. 


2.3.6 Source code Management 


Question< If there are any repository tool being used / managed 
through Dev libraries, then please provide the reference link 


along with the path which specifies how source code versioning is 
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managed across the delivery. Or provide the details around how 
the feature files, Dev libraries, QA automation script library will 
be managed under project folders. Also provide the details around 
how the version controlling will be achieved for all source code 
(feature files, QA automation scripts, libraries etc.). > 


Answer:- Repository Tool and Source Code Versioning 

Repository Tool 

For managing source code versioning, a popular and widely used tool is 
Git. Git is a distributed version control system that allows multiple 
developers to work on the same project simultaneously. Here are some 


key tools and platforms that utilize Git: 
1. GitHub: 


o URL: GitHub 


o Features: Provides a web-based interface for Git repositories, 


supports collaboration, pull requests, and issue tracking. 
2. GitLab: 


o URL: GitLab 


o Features: Offers integrated CI/CD pipelines, project management 


tools, and robust security features. 
3. Bitbucket: 


o URL: Bitbucket 
o Features: Supports Git and Mercurial repositories, integrates with 


Jira for project management, and provides CI/CD capabilities. 
Project Folder Structure and Management 
1. Feature Files: 
o Path: src/main/features/ 
o Description: Contains all the feature files, organized b 
functionality or module. 
o Version Control: Managed using Git, with each feature file 
tracked for changes and updates. 
2. Dev Libraries: 
o Path: src/main/libs/ 
o Description: Contains custom libraries and dependencies used b 
the application. 
o Version Control: Libraries are versioned to ensure compatibility 
and traceability. 
3. QA Automation Script Library: 
o Path: src/test/automation/ 


©2016 Infosys Limited, India 


Infosys Test Strategy for Continuous Validation 


o Description: Contains automated test scripts for functional, 
integration, and performance testing. 

o Version Control: Scripts are versioned to maintain consistency 
and track changes over time. 

Version Controlling Strategy 
1. Branching Strategy: 

o Main Branch: The main branch (e.g., main or master) contains the 
stable version of the code. 

o Feature Branches: Each new feature or bug fix is developed in a 
separate branch (e.g., feature/feature-name). 

o Merge Requests: Changes are merged into the main branch 
through pull requests or merge requests, ensuring code reviews 
and approvals. 

2. Commit Messages: 

o Standards: Use clear and descriptive commit messages to explain 
the changes made. 

o Format: Follow a consistent format, such as type(scope): 
description (e.g., feat(login): add user authentication). 

3. Continuous Integration (CI): 

o Tools: Use Cl tools like Jenkins, GitHub Actions, or GitLab CI to 
automate the build and testing process. 

o Pipeline: Set up pipelines to run automated tests, build the 
application, and deploy to staging or production environments. 

4. Tagging and Releases: 

o Tags: Use tags to mark specific versions or releases (e.g., v1.0.0). 

o Releases: Create release notes to document changes, new 
features, and bug fixes for each version. 

5. Code Reviews: 

o Process: Implement a code review process to ensure code qualit 
and adherence to standards. 

o Tools: Use tools like GitHub Pull Requests or GitLab Merge 
Requests for code reviews. 

By following this structured approach, you can effectively manage and 
version control all source code, feature files, QA automation scripts, 
and libraries, ensuring a smooth and efficient development process. 
Would you like more details on any specific aspect of this strategy? 


2.4 Test types and Phases 
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Following is the list of various testing that needs to be carried out 
as part of the testing along with the objective of each test and the 
characteristics 


= Validation of Acceptance 
e Build verification or Smoke Test 
= Validation of Tasks implementing the User Stories 
e Functional Testing 
e Regression testing <Provide the details of how regression 
test cases are identified, frequency of the execution is 
planned etc > 


Answer:- Identifying Regression Test Cases 


Steps to Identify Regression Test Cases 


1. Analyze Recent Changes: 
o Identify areas of the application that have been modified, includin 
new features, bug fixes, and enhancements. 
o Focus on modules and components that are directly impacted by 
these changes. 


2. Review Critical Functionalities: 
o Identify core functionalities that are critical to the application’s 
Operation. 
o Ensure these functionalities are included in the regression test 
suite to prevent any disruptions. 
3. Assess High-Risk Areas: 
o Determine areas of the application that are prone to defects or 
have a history of frequent issues. 
o Prioritize these areas for regression testing to mitigate potential 
risks. 
4. Leverage Test Coverage Reports: 
o Use test coverage tools to identify gaps in existing test cases. 
o Ensure comprehensive coverage of all impacted areas by adding or 
updating test cases as needed. 
5. Consult with Stakeholders: 
o Collaborate with developers, QA engineers, and product owners to 
identify critical test cases. 
o Incorporate feedback from stakeholders to ensure all important 
scenarios are covered. 


Planning Frequency of Regression Test Execution 
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Factors to Consider 


1. Release Cycle: 
o Align regression testing frequency with the application’s release 
cycle. 
o For frequent releases, consider running regression tests before 
each release. 
2. Development Methodology: 
o In Agile environments, run regression tests at the end of each 
sprint or iteration. 
o For continuous delivery, integrate regression tests into the CI/CD 
pipeline for regular execution. 


3. Change Impact: 
o Assess the impact of recent changes on the application. 
o For significant changes, run a full regression test suite; for minor 
changes, run targeted regression tests. 
4. Resource Availability: 
o Consider the availability of testing resources, including time, tools, 
and personnel. 
o Optimize the regression test suite to balance thoroughness with 
efficiency. 
Recommended Frequency 
1. Daily: 
o For continuous integration and delivery environments, run 
automated regression tests daily. 
o Ensure quick feedback on code changes and maintain application 
stability. 
2. End of Sprint: 
o In Agile development, run regression tests at the end of each 
sprint. 
o Validate that new features and changes have not introduced any 
regressions. 
3. Pre-Release: 
o Run acomprehensive regression test suite before each major 
release. 
o Ensure all critical functionalities are working as expected and no 
new defects have been introduced. 
Tools and Best Practices 


Tools for Regression Testing 
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1. Selenium: 
Automates web application testing and supports multiple browsers 


and platforms. 
o Ideal for functional and regression testing of web applications. 
2. JUnit: 
o A widely used testing framework for Java applications. 


o Supports unit and regression testing with easy integration into 
CI/CD pipelines. 
3. TestNG: 
o A testing framework inspired by JUnit, with additional features for 
parallel test execution and configuration. 
o Suitable for both unit and regression testing. 
4. Jenkins: 
o An open-source automation server that supports CI/CD pipelines. 
o Integrates with various testing tools to automate regression test 
execution. 
Best Practices 


1. Automate Regression Tests: 
o Automate as many regression tests as possible to ensure 
consistency and efficiency. 
o Use tools like Selenium, JUnit, and TestNG for automation. 
2. Maintain Test Suites: 
o Regularly update and maintain regression test suites to reflect 
changes in the application. 
o Remove obsolete test cases and add new ones as needed. 
3. Prioritize Test Cases: 
o Prioritize test cases based on criticality and risk. 
o Focus on high-impact areas to ensure thorough validation. 
4. Monitor Test Results: 
o Continuously monitor and analyze test results to identify trends 
and areas for improvement. 
o Use reporting tools to generate insights and track test coverage. 
By following these strategies and best practices, you can 
effectively identify and manage regression test cases, ensuring 
the stability and reliability of your application. 
e—- Middleware Testing through Automation or 
Virtualization 
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e Performance Testing 

e <Provide any other testing phases to be used> 
Answer:- Additional Testing Phases 
In addition to the standard phases of testing, there are several 
other important testing phases that can be used to ensure 
comprehensive coverage and quality assurance for your 
application. Here are some key additional testing phases: 

1. Unit Testing: 


o Purpose: To test individual components or units of the application 
in isolation. 


o Focus: Ensures that each unit functions correctly. 
o Tools: JUnit, NUnit, TestNG. 


2. Integration Testing: 
o Purpose: To test the interaction between integrated units or 


components. 


o Focus: Ensures that combined components work together as 
expected. 


o Tools: JUnit, TestNG, Postman for API integration. 
3. System Testing: 
o Purpose: To test the complete and integrated application as a 
whole. 
o Focus: Validates the end-to-end functionality of the application. 
o Tools: Selenium, OTP, TestComplete. 


4. Acceptance Testing: 
o Purpose: To verify that the application meets the business 


requirements and is ready for deployment. 
o Focus: Ensures that the application is acceptable to the end-users. 


o Tools: Cucumber, FitNesse. 
9. Performance Testing: 
o Purpose: To assess the application’s performance under various 
conditions. 
o Focus: Fvaluates response times, throughput, and resource usage. 
o Tools: JMeter, LoadRunner, Gatling. 


6. Security Testing: 
o Purpose: To identify vulnerabilities and ensure the application is 
secure. 


o Focus: Protects against threats and ensures data integrity. 
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o Tools: OWASP ZAP, Burp Suite, Nessus. 
7. Usability Testing: 
o Purpose: To evaluate the application’s user interface and user 
experience. 
o Focus: Ensures the application is user-friendly and intuitive. 


o Tools: UserTesting, Crazy E Hotjar. 
8. Compatibility Testing: 
o Purpose: To ensure the application works across different devices 
browsers, and operating systems. 
o Focus: Validates compatibility with various environments. 
o Tools: BrowserStack, Sauce Labs, CrossBrowserTesting. 
9. Regression Testing: 
o Purpose: To verify that recent changes have not adversely affected 
existing functionality. 
o Focus: Ensures stability and reliability after updates. 
o Tools: Selenium, TestNG, Jenkins for automation. 


10. Exploratory Testing: 
o Purpose: To explore the application without predefined test cases. 


o Focus: Identifies unexpected issues and improves overall quality. 


o Tools: Session-based test management (SBTM) tools. 
168 Alpha and Beta Testing: 

o Alpha Testing: Conducted by internal teams to identify bugs 
before releasing to external users. 

o Beta Testing: Conducted by a limited group of external users to 
gather feedback and identify issues in a real-world environment. 
By incorporating these additional testing phases, you can ensure a 
thorough and comprehensive testing process, leading to a higher 
quality and more reliable application. 
Would you like more details on any specific testing phase or tool? 


25 Setting up Automated Gating criteria 
Gating will be implemented for the following - 


Code Quality Gate 

Unit Test Gate 

Functional Smoke Test Gate 
Functional Testing Gate 
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SOA Testing Gate 

Performance Smoke Testing Gate 
Load Testing Gate 

Stress Testing Gate 

Endurance Testing Gate 
Deployment Gate 


Automated Gating will be implemented (expect PT) and the following 
type of gating shall be implemented for all phases - 


Count of Critical / High 
Severity Code Quality 


Defects [ssues greater than 


Gating threshold value - Do 


est Cases Pass % less 
than threshold value 
Pass% Gating |(70% for most gating) 
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2.6  #Risk and Assumption 


< Provide the details of risk and assumption for the current 
application in the context of continuous validation > 


Answer:- Risks and Assumptions for Continuous Validation in the 
Current Application 
Risks 
1. Incomplete Test Coverage: 
o Risk: Not all scenarios and edge cases are covered by automated 
tests. 


o Mitigation: Regularly review and update test cases to ensure 

comprehensive coverage. 
2. False Positives/Negatives: 

o Risk: Automated tests may produce false positives (indicating 
issues where there are none) or false negatives (failing to detect 
actual issues). 

o Mitigation: Continuously refine test scripts and validation criteria 
to improve accuracy. 

3. Tool Integration Issues: 

o Risk: Integration of continuous validation tools with existing CI/CD 
pipelines may face compatibility issues. 

o Mitigation: Conduct thorough compatibility testing and use well- 
documented tools with strong community support. 


4. Performance Overhead: 
o Risk: Continuous validation processes may introduce performance 


overhead, slowing down the development pipeline. 
o Mitigation: Optimize validation scripts and run performance- 
intensive tests during off-peak hours. 
o. Security Vulnerabilities: 
o Risk: Automated validation processes may inadvertently expose 
security vulnerabilities. 
o Mitigation: Implement security best practices and conduct regular 
security audits. 
6. Data Integrity Issues: 
o Risk: Continuous validation may lead to data corruption or loss if 
not properly managed. 
o Mitigation: Use robust data management practices and backup 
strategies. 
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Assumptions 
1. Stable Test Environment: 


Assumption: The test environment is stable and mirrors the 


production environment closely. 
o Impact: Ensures that test results are reliable and reflective of real- 
world scenarios. 
2. Availability of Resources: 
o Assumption: Sufficient resources (e.g., hardware, software, 
personnel) are available to support continuous validation. 
o Impact: Enables smooth execution of validation processes without 
resource constraints. 
3. Tool Compatibility: 
o Assumption: The chosen validation tools are compatible with the 
existing technology stack. 
o Impact: Facilitates seamless integration and reduces the risk of 
tool-related issues. 
4. Regular Updates: 
o Assumption: Test cases and validation scripts are regularly 
updated to reflect changes in the application. 
o Impact: Ensures that validation processes remain relevant and 
effective. 
9. Stakeholder Buy-In: 
o Assumption: All stakeholders (developers, QA, management) are 
committed to the continuous validation process. 
o Impact: Promotes collaboration and ensures that validation efforts 


are supported and prioritized. 
6. Automated Processes: 
Assumption: Most validation processes are automated to reduce 


manual intervention and increase efficiency. 
o Impact: Enhances the speed and reliability of validation efforts. 
By understanding and addressing these risks and assumptions, you can 
ensure a more effective and reliable continuous validation process for your 
application. 
Would you like more details on any specific risk or assumption? 
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3. Transition from old methodology to Continuous validation 


The approach to transition from old methodology to or up to DevOps - 
continuous validation is extreme critical factor for a successful 
transition. 


The traditional planned approach to organization change 
management is replaced by an adaptive, strategy based roll out of 
continuous delivery efforts. The following approach has been adopted 
for the <application name> transition. 


3.1 Objective 


< Provide the details of why transition from old methodology to 
DevOps / Continuous Validation is required. List down key 
objective / performance improvement parameters which are 
commonly found during each deployment. This should be found 
out during gap analysis.> 
Answer:- Transition from Old Methodology to DevOps/Continuous 
Validation 
Why Transition is Required 

1. Improved Code Quality: 

o Continuous validation ensures that code is constantly tested for 
functionality, security, and performance, leading to higher quality 
code!. 

2. Early Detection of Issues: 

o By validating code continuously, potential issues are identified and 

resolved early in the development cycle, reducing the risk of major 


defects!. 
3. Enhanced Compliance: 
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o Continuous validation helps in maintaining compliance with 
industry standards and regulations by regularly checking code 
against these requirements’. 

4. Increased Efficiency: 

o Automating the validation process reduces the need for manual 

checks, allowing developers to focus on more value-added tasks’. 
9. Better Alignment with Business Goals: 

o Continuous validation ensures that the development efforts are 
aligned with business objectives by regularly reviewing and 
validating the strategy’. 

Key Objectives/Performance Improvement Parameters 
1. Reduced Deployment Risks: 

o Continuous validation minimizes the risk of deploying faulty code, 

leading to more stable and reliable releases’. 
2. Faster Time-to-Market: 

o With automated validation, the development process becomes more 

efficient, enabling quicker releases and updates’. 
3. Improved Customer Satisfaction: 

o Higher quality and more reliable applications result in better user 

experiences and increased customer satisfaction’. 
4. Cost Savings: 

o Early detection and resolution of issues reduce the cost associated 

with fixing defects later in the development cycle’. 
o. Scalability: 

o Continuous validation supports scalable development practices, 
allowing the application to grow and adapt to new challenges and 
requirements’. 

By transitioning to DevOps and continuous validation, your application 
can achieve these objectives, leading to a more efficient and reliable 
development process. 

Would you like more details on any specific aspect of this transition? ': 1 


3.2 Assess current capability 


< Provide the details of the current capability of the delivery 
process, key skills of the DevOps - Continuous validation team, 
Internal Continuous validation expert etc. > 
Answer:- Current Capability of the Delivery Process 
Current Capability 

1. Automated CI/CD Pipeline: 
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o Integration: Continuous Integration (CI) tools like Jenkins, GitHub 


Actions, or GitLab CI are used to automate the build and testing 
process. 

o Deployment: Continuous Deployment (CD) ensures that code 
changes are automatically deployed to staging or production 
environments after passing all tests’. 

2. Comprehensive Testing: 

o Unit Testing: Automated unit tests validate individual 
components. 

o Integration Testing: Ensures that integrated components work 
together as expected. 

o System Testing: Validates the entire system’s functionality. 


o Performance Testing: Assesses the application’s performance 
under various conditions!. 
3. Monitoring and Logging: 
o Real-Time Monitoring: Tools like Prometheus and Grafana are 
used to monitor application performance. 
o Centralized Logging: ELK Stack (Elasticsearch, Logstash, 
Kibana) is used for centralized logging and analysis’. 


4. Security: 
Automated Security Scans: Tools like OWASP ZAP and Bur 


Suite are used for regular security assessments. 
o Compliance: Continuous validation ensures compliance with 
industry standards and regulations’. 
Key Skills of the DevOps - Continuous Validation Team 
1. Automation Expertise: 
o Scripting: Proficiency in scripting lanquages like Python, Bash, or 
PowerShell for automation tasks. 


o CI/CD Tools: Experience with CI/CD tools like Jenkins, GitHub 
Actions, GitLab CI, and CircleCI?. 


2. Testing Skills: 
Automated Testing: Knowledge of automated testing frameworks 


like Selenium, JUnit, TestNG, and Cypress. 
o Performance Testing: Skills in tools like JMeter and LoadRunner 
for performance testing’. 
3. Infrastructure Management: 
o Infrastructure as Code (IaC): Experience with tools like 


Terraform, Ansible, and CloudFormation for managing 
infrastructure. 


©2016 Infosys Limited, India 


Infosys Test Strategy for Continuous Validation 


o Containerization: Proficiency in Docker and Kubernetes for 
containerization and orchestration’. 


4. Monitoring and Logging: 
o Monitoring Tools: Experience with Prometheus, Grafana, and 
Nagios for monitoring. 
o Logging Tools: Skills in using ELK Stack for centralized logging”. 
o. Security: 
o Security Testing: Knowledge of security testing tools like OWASP 
ZAP and Burp Suite. 
o Compliance: Understanding of security standards and compliance 
requirements’. 
Internal Continuous Validation Expert 
1. Role and Responsibilities: 
o Strategy Development: Develop and implement continuous 
validation strategies to ensure code quality and compliance. 
o Tool Integration: Integrate continuous validation tools into the 
CI/CD pipeline. 
o Training and Mentorship: Provide training and mentorship to 
the team on best practices for continuous validation®. 
2. Skills and Expertise: 
o Deep Understanding of CI/CD: Expertise in continuous 
integration and continuous deployment practices. 
o Automation Proficiency: Strong skills in automation tools and 
scripting languages. 
o Testing Knowledge: Extensive experience in automated testing 
frameworks and performance testing tools. 
o Security Awareness: Knowledge of security testing and 
compliance standards?. 
By leveraging these capabilities and skills, the DevOps - Continuous 
Validation team can ensure a robust, efficient, and secure delivery 
process for your application. 


Would you like more details on any specific aspect of this transition? ': 10 
2,4 °:2 


3.3 Capability gaps, areas of improvement and priority 


< Identify and provide the capability gap, improvement scope to 
bridge the capability gap and priority order to achieve them. > 
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Answer:- Capability Gap Analysis for Continuous Validation 
Transition 
Current Capability Gaps 
1. Manual Testing: 
o Gap: Testing is performed manually, leading to inconsistencies and 
potential human errors. 
o Impact: Slower feedback loops and higher chances of undetected 
defects. 
2. Delayed Issue Detection: 
o Gap: Issues are often detected late in the development cycle. 
o Impact: Increased cost and effort required to fix defects. 
3. Limited Automation: 
o Gap: Limited or no automation in the validation process. 
o Impact: Slower development cycles and inefficient resource 
utilization. 
4. Inconsistent Compliance: 
o Gap: Difficulty in maintaining consistent compliance with indust 
standards and regulations. 
o Impact: Potential legal and requlatory risks. 
5. Resource Intensive: 
o Gap: Manual validation is resource-intensive, requiring significant 
time and effort from the development team. 
o Impact: Reduced productivity and higher operational costs. 
Improvement Scope to Bridge the Capability Gaps 


1. Implement Automated Testing: 
Action: Develop and integrate automated test scripts for various 


validation scenarios (unit tests, integration tests, performance 


tests). 
o Tools: Use tools like Selenium, JUnit, TestNG, and Jenkins for 
automation. 


o Benefit: Ensures consistent and reliable validation, reducing 
manual effort and errors. 
2. Early Issue Detection: 
o Action: Integrate continuous validation into the CI/CD pipeline to 
detect and resolve issues early. 
o Tools: Use CI tools like Jenkins, GitHub Actions, or GitLab CI. 
o Benefit: Reduces the risk of major defects and lowers the cost of 


fixing issues. 
3. Enhanced Automation: 
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o Action: Leverage automation tools to streamline the validation 
process and provide faster feedback. 
o Tools: Use tools like Selenium, JUnit, and TestNG. 
o Benefit: Increases efficiency and accelerates development cycles. 
4. Consistent Compliance: 
o Action: Define and implement automated checks for compliance 
with industry standards and regulations. 
o Tools: Use tools like OWASP ZAP for security testing and 
compliance checks. 
o Benefit: Ensures consistent compliance and reduces legal and 


regulatory risks. 
5. Efficient Resource Utilization: 


o Action: Automate the validation process to reduce manual effort 
and allow the team to focus on more critical tasks. 
o Tools: Use automation frameworks and CI/CD tools. 
o Benefit: Increases productivity and reduces operational costs. 
Priority Order to Achieve Improvements 
1. Implement Automated Testing: 
o Priority: High 
o Reason: Provides immediate benefits in terms of consistenc 
reliability, and reduced manual effort. 
2. Early Issue Detection: 
o Priority: High 
o Reason: Critical for reducing the risk of major defects and 
lowering the cost of fixing issues. 
3. Enhanced Automation: 
o Priority: Medium 
o Reason: Improves efficiency and accelerates development cycles 
but requires initial setup and integration. 
4. Consistent Compliance: 
o Priority: Medium 
o Reason: Ensures legal and regulatory compliance, reducin 
potential risks. 
5. Efficient Resource Utilization: 
o Priority: Low 
o Reason: While important, it is a byproduct of implementin 
automated testing and enhanced automation. 
By addressing these capability gaps in the specified priority order, you 
can effectively transition to continuous validation, leading to a more 


efficient, reliable, and compliant development process. 
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Would you like more details on any specific improvement action or tool? 


3.4 Roadmap of milestone and execution timeline 


< Provide the details of the roadmap prepared for milestones 
defined, mechanism to track the execution of the milestone 
against the time and road ahead> 


Answer:- Roadmap for Transition to Continuous Validation 
Milestones Defined 


1. Initial Assessment and Planning: 
o Objective: Conduct a thorough assessment of the current 


validation processes and identify key areas for improvement. 
o Timeline: 1 month 
o Deliverables: Detailed assessment report, implementation plan 
resource allocation. 
2. Tool Selection and Integration: 
o Objective: Evaluate and select appropriate automation and 
validation tools. 
o Timeline: 2 months 
o Deliverables: List of selected tools, integration plan, initial setup. 
3. Automated Test Development: 
o Objective: Develop automated test scripts for various validation 
scenarios (unit tests, integration tests, performance tests). 
o Timeline: 3 months 


o Deliverables: Automated test scripts, test cases repository, initial 
test runs. 
4. CI/CD Pipeline Integration: 
o Objective: Integrate continuous validation into the CI/CD pipeline. 
o Timeline: 2 months 
o Deliverables: Updated CI/CD pipeline, automated test execution, 
monitoring setup. 
5. Compliance and Security Checks: 
o Objective: Implement automated checks for compliance with 


industry standards and security requirements. 
o Timeline: 2 months 
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o Deliverables: Compliance check scripts, security validation 
reports, regular audits. 
6. Training and Support: 
o Objective: Provide training to the development team on new 
validation processes and tools. 
o Timeline: 1 month 
o Deliverables: Training sessions, documentation, support system. 
7. Continuous Improvement: 
o Objective: Regularly review and refine the continuous validation 


process. 
Timeline: Ongoin 


o Deliverables: Updated validation processes, feedback 


incorporation, process optimization. 
Mechanism to Track Execution of Milestones 
1. Project Management Tools: 
o Tools: Use project management tools like Jira, Trello, or Asana to 
track progress. 
o Features: Task assignments, progress tracking, milestone 
tracking, and reporting. 
2. Regular Status Meetings: 
o Frequency: Weekly or bi-weekly meetings to review progress and 
address any issues. 
o Participants: Project stakeholders, development team, QA team, 
and management. 
3. Progress Reports: 
o Frequency: Monthly progress reports to provide updates on 
milestone achievements and any deviations from the plan. 
o Content: Status of each milestone, completed tasks, upcoming 
tasks, and any risks or issues. 
4. Dashboards and Metrics: 
o Tools: Use dashboards in project management tools to visualize 
progress. 
o Metrics: Track key performance indicators (KPIs) such as test 
coverage, defect rates, and deployment frequency. 
9. Feedback Loops: 
o Mechanism: Reqular feedback sessions with the development and 
QA teams to gather insights and make necessary adjustments. 


o Outcome: Continuous improvement of the validation process 
based on feedback. 


Road Ahead 
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1. Scalability: 
o Objective: Ensure the validation process can scale with the 
application’s growth. 
o Actions: Optimize automation scripts, enhance infrastructure, and 


continuously monitor performance. 
2. Advanced Automation: 


o Objective: Implement advanced automation techniques such as AI- 


driven testing and self-healing scripts. 
o Actions: Research and integrate advanced automation tools, 
conduct pilot projects, and scale successful implementations. 
3. Enhanced Security: 
o Objective: Strengthen security validation to protect against 
evolving threats. 
o Actions: Regularly update security checks, conduct penetration 
testing, and stay informed about new security practices. 
4. User Experience Focus: 
o Objective: Improve the user experience by incorporating usabilit 
testing into the validation process. 
o Actions: Develop and integrate usability test cases, gather user 
feedback, and iterate on improvements. 
By following this roadmap and tracking mechanisms, you can ensure a 
successful transition to continuous validation, leading to improved code 
quality, faster time-to-market, and enhanced customer satisfaction. 


Would you like more details on any specific milestone or tracking 
mechanism? 
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4. An illustrative list of activities of a continuous validation 
engineer 


The following summarizes the typical activities to be done by DevOps 
team member during different sprints of the continuous delivery 
model project. 


Sprint 0 Sprint N-1 Sprint N / Release 


Hard Sprint 


Prepare Test 
Strategy Doc 
Participate in 
Requirement 
analysis 
Automation 
Roadmap 


Preparation 
of Feature 
file 


Development 
of 
Automation 
scripts 
Source code 
management 
Daily Scrum 
Meeting 
Automated 
Execution of 
various test 
phases 
Functionality 
demonstratio 
n & Sign off 


Final System 
Testing 

Full 
Regression 
Testing 
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Sequence of 
Events 
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5. Continuous Validation - Building Pipeline 


5.1 In Scope 


< Provide the details of which phases of the testing (based on the 
architecture of the application) and the reason to be included in 
the continuous validation pipeline > 


Answer:- Phases of Testing to Include in the Continuous Validation 
Pipeline 
1. Unit Testing 

- Purpose: To test individual components or units of the application in 
isolation. 

« Reason: Unit tests are the fastest and least expensive to run. They 
ensure that each unit functions correctly, catching issues early in the 
development process’. 

2. Integration Testing 

- Purpose: To test the interaction between integrated units or 
components. 

« Reason: Ensures that combined components work together as 


expected, identifying issues that may arise from component 
interactions!. 


3. System Testing 
- Purpose: To test the complete and integrated application as a whole. 
« Reason: Validates the end-to-end functionality of the application, 
ensuring that all components work together seamlessly’. 
4. Regression Testing 
- Purpose: To verify that recent changes have not adversely affected 
existing functionality. 
« Reason: Ensures stability and reliability after updates, preventing new 
changes from breaking previously working features’. 
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3. Performance Testing 
- Purpose: To assess the application’s performance under various 
conditions. 
« Reason: Identifies bottlenecks and optimizes performance, ensuring the 
application can handle expected load and stress”. 
6. Security Testing 
- Purpose: To identify vulnerabilities and ensure the application is 
secure. 
« Reason: Protects against threats and ensures data integrity, 
maintaining compliance with security standards?®. 
7. User Acceptance Testing (UAT 


- Purpose: To validate the application against user requirements and 
ensure it meets business needs. 
« Reason: Ensures the application is acceptable to end-users and meets 
their expectations’. 
Reasons to Include These Phases in the Continuous Validation 
Pipeline 
1. Early Defect Detection: 

o Continuous validation allows for early detection and resolution of 
issues, reducing the risk of major defects and lowering the cost of 
fixing issues*. 

2. Improved Code Quality: 

o By continuously validating code, the overall quality of the 
application is improved, leading to more stable and reliable 
releases?. 

3. Faster Feedback Loops: 

o Automated testing provides rapid feedback to developers, enabling 

quicker identification and resolution of issues’. 
4. Enhanced Compliance: 

o Continuous validation helps maintain compliance with industry 
standards and regulations by regularly checking code against these 
requirements’. 

9. Increased Efficiency: 

o Automating the validation process reduces the need for manual 

checks, allowing developers to focus on more value-added tasks?. 
6. Better Alignment with Business Goals: 

o Continuous validation ensures that the development efforts are 
aligned with business objectives by regularly reviewing and 
validating the strategy”. 
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By including these testing phases in the continuous validation pipeline, 
you can ensure a comprehensive and efficient validation process, leading 
to higher quality and more reliable application releases. 


Would you like more details on any specific testing phase or aspect of 
continuous validation? !: 1 7: 2 °: 4 


5.11 Test Data Setup 


< Provide the details around how test data set up will be 
automated. Also provide the details of tools being used, which 
activities to be carried (refresh database, running query etc) > 


Answer:- Automating Test Data Setup 


Automated Test Data Setup 
1. Data Generation: 


o Automated Tools: Use tools like Tonic.ai, Mockaroo, or 


DataFactory to generate synthetic test data. 

o Activities: Create diverse datasets including valid, invalid, edge 
cases, and boundary values to ensure comprehensive test 
coverage’. 


2. Data Management: 
Automated Tools: Use tools like Delphix, Informatica TDM, or CA 


Test Data Manager for managing test data. 
o Activities: Organize and store test data in a structured manner, 
ensuring easy access and version control’. 


3. Data Masking and Anonymization: 
Automated Tools: Use tools like IBM InfoSphere Optim, DataVeil 


or Oracle Data Masking. 
o Activities: Mask or anonymize sensitive data to protect privacy 
and comply with data protection regulations’. 
4. Data Refresh and Cleanup: 
o Automated Tools: Use scripts or tools like Jenkins, Ansible, or 
Chef to automate data refresh and cleanup. 
o Activities: Refresh databases, reset test data between test runs 


and clean up test data after testing is complete’. 
5. Data Validation: 


o Automated Tools: Use tools like QuerySurge or custom scripts for 


data validation. 
o Activities: Perform automated checks to validate the correctness 


and integrity of test data’. 
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Tools Used for Test Data Automation 
1. Tonic.ai: 
o Purpose: Generate and manage synthetic test data. 
o Features: Data generation, masking, and variation’. 
2. Mockaroo: 
o Purpose: Generate realistic test data for various scenarios. 


o Features: Customizable data generation, support for multiple data 
types’. 
3. Delphix: 
o Purpose: Manage and provision test data. 
o Features: Data masking, data virtualization, and automated data 
refresh!. 
4. Informatica TDM: 
o Purpose: Test data management and provisioning. 


o Features: Data generation, masking, and subsetting’. 
o. CA Test Data Manager: 
o Purpose: Comprehensive test data management. 
o Features: Data generation, masking, and provisioning’. 
Activities to be Carried Out 
1. Refresh Database: 


o Tools: Jenkins, Ansible, Chef. 
o Activities: Automate the process of refreshing the database to 
ensure it is up-to-date with the latest test data’. 
2. Running Queries: 
o Tools: Custom scripts, SOL tools. 
o Activities: Execute queries to validate data, generate reports, and 
ensure data integrity’. 
3. Data Masking: 
o Tools: IBM InfoSphere Optim, DataVeil. 
o Activities: Mask sensitive data to protect privacy and comply with 
regulations’. 
4. Data Cleanup: 
o Tools: Custom scripts, Jenkins. 
o Activities: Clean up test data after testing to maintain a consistent 
test environment!. 
By automating these activities and using the appropriate tools, you can 
ensure a robust and efficient test data setup process, leading to more 
reliable and comprehensive testing. 
Would you like more details on any specific tool or activity? *: 1 


©2016 Infosys Limited, India 


Infosys Test Strategy for Continuous Validation 


5.1.2 Smoke Testing 


< Provide the details of which functionality is being tested during 
smoke testing. Also brief how specific functionality ensures the 
smoke test of the application. Provide the details of the tools 
being used for automating smoke test. > 


Answer:- Functionality Tested During Smoke Testing 
Key Functionalities Tested 


1. User Authentication: 
o Details: Verifies that users can log in and log out successfully. 
o Reason: Ensures that the authentication mechanism is workin 


correctly, which is critical for accessing the application. 
2. Navigation: 
o Details: Checks that users can navigate through the main sections 
of the application. 
o Reason: Ensures that the application’s navigation is functional and 
intuitive, allowing users to access different features. 
3. Data Input and Output: 
o Details: Tests forms and data ent oints to ensure data can be 
entered and retrieved correctly. 
o Reason: Validates that the application can handle user inputs and 
display outputs as expected. 


4. Core Functionalities: 


o Details: Verifies that the primary features of the application (e.g., 
adding items to a cart, processing transactions) are working. 
o Reason: Ensures that the main functionalities are operational, 
which is essential for the application’s purpose. 
o. Error Handling: 
o Details: Checks that the application handles errors gracefully and 


provides appropriate error messages. 


o Reason: Ensures that users receive clear feedback when 


something goes wrong, improving the user experience. 
6. Database Connectivity: 


o Details: Verifies that the application can connect to the database 


and perform basic CRUD (Create, Read, Update, Delete) 


operations. 
o Reason: Ensures that the application can interact with the 
database correctly, which is crucial for data integrity. 
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How Specific Functionality Ensures the Smoke Test of the 
Application 
1. User Authentication: 
o Ensures: Validates that users can access the application securel 


which is a fundamental requirement for any application. 
2. Navigation: 
o Ensures: Confirms that users can move through the application 
without issues, which is essential for usability. 
3. Data Input and Output: 
o Ensures: Verifies that the application can handle user data 
correctly, which is critical for functionality. 
4. Core Functionalities: 
o Ensures: Confirms that the main features of the application are 


working, which is essential for the application’s purpose. 
o. Error Handling: 
o Ensures: Validates that the application can handle errors 
gracefully, which is important for user experience. 
6. Database Connectivity: 
o Ensures: Confirms that the application can interact with the 
database, which is crucial for data integrity and functionality. 


Tools Used for Automating Smoke Tests 
1. Selenium: 


o Purpose: Automates web application testing. 


o Features: Supports multiple browsers and platforms, integrates 
with CI/CD pipelines’. 
2. Cypress: 
o Purpose: End-to-end testing for web applications. 
o Features: Fast, reliable, and easy to set up, with real-time 
reloading’. 
3. JUnit/TestNG: 
o Purpose: Frameworks for writing and running repeatable 
automated tests in Java applications. 


o Features: Supports unit, integration, and smoke testing’. 
4. Postman: 


o Purpose: API testing and automation. 


o Features: Easy to create and run automated tests for APIs, 
integrates with CI/CD pipelines’. 
By using these tools and focusing on the key functionalities, you can 
ensure that your smoke tests effectively validate the stability and 


readiness of your application for further testing. 
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Would you like more details on any specific tool or functionality? *: 1 7: 2 


5.1.3 Functional Testing 


< Brief about the functionalities to be tested during the 
continuous validation. Provide the details around tools being used 
to develop the automation scripts. > 
Answer:- Functionalities to be Tested During Continuous 
Validation 
Key Functionalities 

1. Core Business Logic: 

o Details: Ensures that the main functionalities of the application 
such as transaction processing, data calculations, and business 
rules, are working correctly. 

o Reason: Validates that the core operations of the application are 
reliable and accurate. 

2. User Interface (UI): 

o Details: Tests the responsiveness, usability, and accessibility of 
the application’s UI components. 

o Reason: Ensures a seamless and user-friendly experience across 


different devices and browsers. 


3. API Endpoints: 
o Details: Validates the functionalit erformance, and security of 
API endpoints. 
o Reason: Ensures that the application can communicate effectively 
with other services and systems. 
4. Database Operations: 
o Details: Tests CRUD (Create, Read, Update, Delete) operations 
and data integrity. 
o Reason: Ensures that the application can interact with the 
database correctly and maintain data consistency. 
9. Security Features: 
o Details: Validates authentication, authorization, and data 
encryption mechanisms. 
o Reason: Ensures that the application is secure and protects user 


data. 
6. Performance Metrics: 


o Details: Assesses the application’s performance under various load 


conditions. 
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o Reason: Identifies bottlenecks and optimizes performance to 
handle expected traffic. 
7. Error Handling: 
o Details: Tests how the application handles errors and exceptions. 
o Reason: Ensures that the application provides meaningful error 
messages and recovers gracefully from failures. 


Tools Used for Developing Automation Scripts 
1. Selenium: 


o Purpose: Automates web application testing. 
o Features: Supports multiple browsers and platforms, integrates 
with CI/CD pipelines’. 
2. Cypress: 
o Purpose: End-to-end testing for web applications. 
o Features: Fast, reliable, and easy to set up, with real-time 
reloading’. 
3. JUnit/TestNG: 
o Purpose: Frameworks for writing and running repeatable 
automated tests in Java applications. 


o Features: Supports unit, integration, and regression testing’. 
4. Postman: 


o Purpose: API testing and automation. 
o Features: Easy to create and run automated tests for APIs, 
integrates with CI/CD pipelines’. 


9. Jenkins: 
o Purpose: Continuous integration and continuous delivery (CI/CD) 
automation. 


o Features: Automates the build, test, and deployment processes, 
integrates with various testing tools’. 
6. TestComplete: 
o Purpose: Functional UI testing tool. 
o Features: Supports desktop, web, and mobile applications, 
integrates with CI/CD pipelines’. 
By leveraging these tools and focusing on the key functionalities, you can 
ensure that your continuous validation process is thorough, efficient, and 
reliable. 


Would you like more details on any specific tool or functionality? ': 9 


5.1.4 Middleware testing through Automation and Virtualization 
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< Provide the details how SOA or Middleware testing will be 
carried through automation. Brief about which tool has been used 
for automation, virtualization and which services are virtualized to 
ensure middleware testing is fully covered. > 


Answer:- Automating SOA and Middleware Testing 
Automation of SOA and Middleware Testing 
1. Service-Oriented Architecture (SOA) Testing: 
o Objective: Validate the interactions between services, ensurin 
they work together as expected. 
o Approach: Automate the testing of individual services and their 
integrations using specialized tools. 
2. Middleware Testing: 
o Objective: Ensure that the middleware components (e.g., message 
brokers, application servers) function correctly and efficiently. 
o Approach: Automate the testing of middleware interactions, 


message flows, and data transformations. 
Tools Used for Automation 


1. Parasoft SOAtest: 
o Purpose: Comprehensive testing of SOA and APIs. 


o Features: Supports functional, security, and performance testing 


of web services and APIs’. 


2. SoapuUI: 
o Purpose: Functional testing of web services. 


o Features: Supports SOAP and REST protocols, easy to create and 
run automated tests!. 


3. IBM Rational Integration Tester: 
o Purpose: Automated testing of complex integrations. 
o Features: Supports testing of middleware, APIs, and message 
queues’. 
4. Apache JMeter: 
o Purpose: Performance and load testing. 
o Features: Supports testing of web applications, APIs, and various 


protocols”. 
5. Postman: 


o Purpose: API testing and automation. 
o Features: Easy to create and run automated tests for APIs, 
integrates with CI/CD pipelines’. 


Virtualization for Middleware Testing 
1. Service Virtualization: 
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o Objective: Simulate the behavior of dependent components or 
services to enable testing in isolation. 
o Tools: 
: CA Service Virtualization: Simulates services, databases 
and mainframes to enable testing without dependencies’. 
: Parasoft Virtualize: Creates virtual services to simulate the 
behavior of dependent components’. 
2. Virtualized Services: 
o API Endpoints: Simulate API responses to test how the 
application handles different scenarios. 
o Database Services: Virtualize database interactions to test data 


retrieval and storage without affecting the actual database. 
o Message Queues: Simulate message queues to test message flows 
and processing. 
o Third-Party Services: Virtualize external services to test 
integrations without relying on the actual third-party systems. 
By using these tools and virtualization techniques, you can ensure 
comprehensive and efficient testing of SOA and middleware components, 
leading to more reliable and robust applications. 
Would you like more details on any specific tool or aspect of SOA and 
middleware testing? ': 5 7: 6 4: 11 


5.15 Performance Testing 


< Provide the details of which performance testing aspects - 
Stress, Load and Endurance. Provide the details around 
concurrent users, peak load and tool being used for the 
continuous validation. Brief about how the result will impact the 
functionality of an application > 
Answer:- Performance Testing Aspects 
1. Stress Testing 
- Purpose: Fvaluates the application’s behavior under extreme load 
conditions, beyond its normal operational capacity. 
« Objective: Identify the breaking point and how the application recovers 
from failure. 
- Example: Simulating a sudden spike in user traffic to see how the 
application handles the load. 
2. Load Testing 
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- Purpose: Measures the application’s performance under expected load 
conditions. 


« Objective: Ensure the application can handle the anticipated number of 
concurrent users and transactions. 
- Example: Simulating the average number of users accessing the 
application during peak hours. 
3. Endurance Testing 
- Purpose: Assesses the application’s performance over an extended 


period. 
« Objective: Detect issues like memory leaks or performance degradation 
over time. 


« Example: Running the application continuously for several hours or 
days to monitor its stability and performance”?. 
Concurrent Users and Peak Load 
« Concurrent Users: The number of users accessing the application 
simultaneously. This is crucial for understanding how the application 


performs under real-world usage scenarios. 
« Peak Load: The maximum number of concurrent users or transactions 


the application can handle without performance degradation. 
Tools Used for Continuous Validation 


1. Apache JMeter: 
o Purpose: Performance and load testing. 
o Features: Simulates multiple users, measures response times, and 


identifies performance bottlenecks’. 
2. LoadRunner: 


o Purpose: Load testing and performance monitoring. 


o Features: Supports a wide range of protocols, provides detailed 
performance metrics’. 
3. Gatling: 
o Purpose: Load and stress testing. 
o Features: High performance, easy-to-use DSL for test scripting’. 
Impact of Results on Application Functionality 
1. Identifying Bottlenecks: 
o Impact: Helps pinpoint areas where the application struggles 
under load, allowing for targeted optimizations. 
2. Ensuring Scalability: 
o Impact: Validates that the application can scale to handle 
increased user load, ensuring it can grow with user demand. 
3. Improving Stability: 
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o Impact: Detects issues like memory leaks or performance 
degradation over time, leading to a more stable application. 
4. Enhancing User Experience: 
o Impact: Ensures that the application remains responsive and 


reliable under various load conditions, improving overall user 
satisfaction. 
By incorporating these performance testing aspects and using the 
appropriate tools, you can ensure that your application is robust, 
scalable, and capable of delivering a high-quality user experience. 
Would you like more details on any specific aspect of performance 
testing? ': 1 *: 7 


5.2 Tool Selection Strategy 


< Provide the details of technology fitment with the tool, 
commercial aspect (open source vs licensed), the tools being 
considered, comparison factors and final conclusion of the tools 
being selected.> 


Answer::- Technology Fitment and Tool Selection 
Technology Fitment with the Tool 
1. Compatibility: 
o Details: Ensure the tool supports the technologies used in your 
application (e.g., programming languages, frameworks, databases). 
o Example: Selenium is suitable for web applications, while Appium 
is better for mobile applications. 
2. Integration: 
o Details: Check if the tool integrates well with your existing CI/CD 
pipeline and other tools. 
o Example: Jenkins integrates seamlessly with various testing tools 
like Selenium, JUnit, and Postman. 
3. Scalability: 
o Details: Ensure the tool can handle the scale of your application 
and testing needs. 
o Example: JMeter can simulate thousands of users for load testing. 
Commercial Aspect: Open Source vs. Licensed 


1. Open Source Tools: 
o Pros: 


_ = Cost-effective (free to use). 
= Community support and frequent updates. 
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« Flexibility to customize. 
o Cons: 
» May require more technical expertise. 
» Limited official support. 
» Potential security risks if not managed properly. 


2. Licensed Tools: 
o Pros: 


» Professional support and maintenance. 
= Comprehensive features and integrations. 
= Compliance with industry standards. 

o Cons: 


: Higher cost (license fees). 


= Vendor lock-in. 


» May have less flexibility for customization. 
Tools Being Considered 
1. Selenium (Open Source): 
o Purpose: Web application testing. 
o Features: Supports multiple browsers and platforms, integrates 
with CI/CD tools}. 
2. LoadRunner (Licensed): 
o Purpose: Performance and load testing. 
o Features: Supports a wide range of protocols, detailed 
performance metrics’. 
3. Postman (Open Source): 
o Purpose: API testing and automation. 
o Features: Easy to create and run automated tests, integrates with 
CI/CD pipelines’. 
4. TestComplete (Licensed): 
o Purpose: Functional UI testing. 
o Features: Supports desktop, web, and mobile applications, 
integrates with CI/CD pipelines’. 


Comparison Factors 
1. Cost: 


o Open Source: Free to use, but may incur costs for setup and 
maintenance. 
o Licensed: Upfront and ongoing license fees, but includes support 
and updates. 
2. Support: 
o Open Source: Community support, which can be variable. 


o Licensed: Professional support with SLAs. 
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3. Features: 
o Open Source: Basic features with potential for customization. 
o Licensed: Comprehensive features out-of-the-box. 
4. Ease of Use: 
o Open Source: May require more technical expertise. 
o Licensed: Generally easier to use with better documentation and 
support. 


9. Security: 
o Open Source: Requires careful management to ensure security. 


o Licensed: Often includes built-in security features and compliance. 
Final Conclusion 
Based on the comparison, the choice between open source and licensed 
tools depends on your specific needs and constraints: 

- For Cost-Effectiveness and Flexibility: Open source tools like 
Selenium and Postman are ideal if you have the technical expertise to 
manage and customize them. 

- For Comprehensive Features and Support: Licensed tools like 
LoadRunner and TestComplete are better if you need professional 
support, compliance, and a wide range of features out-of-the-box. 

Ultimately, a hybrid approach, leveraging both open source and licensed 
tools, can provide the best of both worlds, ensuring cost-effectiveness, 
flexibility, and comprehensive support. 

Would you like more details on any specific tool or aspect of this 
comparison? !: 1 


5.3 Test Automation Roadmap 


< Provide the detailed plan of automation script to be delivered 
over various testing phases against the sprints. > 


Answer:- Detailed Plan for Automation Script Delivery Across 
Testing Phases and Sprints 
Sprint 1: Initial Setup and Unit Testing 
- Activities: 
o Setup the automation framework (e.g., Selenium, JUnit). 


o Develop initial unit test scripts for core functionalities. 


o Integrate unit tests into the CI/CD pipeline. 


- Deliverables: 
Automation framework setup. 
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o Unit test scripts for core functionalities. 


o CI/CD integration for unit tests. 
Sprint 2: Integration Testing 
- Activities: 
o Develop integration test scripts to validate interactions between 
components. 
o Automate integration tests and integrate them into the CI/CD 
pipeline. 
o Ensure data setup and teardown processes are automated. 
- Deliverables: 
o Integration test scripts. 
Automated data setup and teardown scripts. 


o CI/CD integration for integration tests. 
Sprint 3: System Testing 
- Activities: 
o Develop system test scripts to validate end-to-end functionality. 
o Automate system tests and integrate them into the CI/CD pipeline. 


o Perform initial system testing and refine scripts based on feedback. 
« Deliverables: 
o System test scripts. 


o CI/CD integration for system tests. 


o Refined system test scripts. 
Sprint 4: Regression Testing 
- Activities: 
o Identify critical test cases for regression testing. 
o Develop and automate regression test scripts. 
o Integrate regression tests into the CI/CD pipeline. 
- Deliverables: 
o Regression test scripts. 


o CI/CD integration for regression tests. 


o Comprehensive regression test suite. 
Sprint 5: Performance Testing 
- Activities: 
o Develop performance test scripts to assess load, stress, and 
endurance. 


o Automate performance tests using tools like JMeter or 


LoadRunner. 


o Integrate performance tests into the CI/CD pipeline. 


« Deliverables: 
o Performance test scripts. 
©2016 Infosys Limited, India 


Infosys Test Strategy for Continuous Validation 


o CI/CD integration for performance tests. 


o Initial performance test results and analysis. 
Sprint 6: Security Testing 
- Activities: 
o Develop security test scripts to identify vulnerabilities. 


o Automate security tests using tools like OWASP ZAP or Burp Suite. 
o Integrate security tests into the CI/CD pipeline. 


« Deliverables: 
o Security test scripts. 
o CI/CD integration for security tests. 


o Initial security test results and analysis. 
Sprint 7: User Acceptance Testing (UAT 


- Activities: 
o Develop UAT scripts based on user requirements. 
o Automate UAT scripts and integrate them into the CI/CD pipeline. 
o Perform UAT and gather feedback from stakeholders. 
¢ Deliverables: 
o UAT scripts. 


o CI/CD integration for UAT. 
o UAT feedback and refinement. 


Tools Being Used for Automation 
1. Selenium: 


o Purpose: Web application testing. 
o Features: Supports multiple browsers and platforms, integrates 
with CI/CD tools!. 
2. JUnit/TestNG: 


o Purpose: Unit and integration testing. 


o Features: Supports repeatable automated tests, integrates with 
CI/CD pipelines’. 


3. Postman: 
o Purpose: API testing and automation. 
o Features: Easy to create and run automated tests for APIs, 
integrates with CI/CD pipelines’. 
4. JMeter: 
o Purpose: Performance testing. 
o Features: Simulates multiple users, measures response times, 


identifies performance bottlenecks’. 
5. OWASP ZAP: 


o Purpose: Security testing. 
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o Features: Identifies vulnerabilities, integrates with CI/CD 
pipelines’. 
Tracking and Execution 
1. Project Management Tools: 
o Tools: Jira, Trello, Asana. 
o Features: Task assignments, progress tracking, milestone 
tracking, and reporting. 
2. CI/CD Integration: 
o Tools: Jenkins, GitHub Actions, GitLab Cl. 
o Features: Automates the build, test, and deployment processes 
integrates with various testing tools. 
3. Regular Status Meetings: 
o Frequency: Weekly or bi-weekly meetings to review progress and 
address any issues. 
o Participants: Project stakeholders, development team, QA team, 
and management. 
4. Progress Reports: 
o Frequency: Monthly progress reports to provide updates on 
milestone achievements and any deviations from the plan. 
o Content: Status of each milestone, completed tasks, upcoming 
tasks, and any risks or issues. 
By following this detailed plan and using the appropriate tools, you can 
ensure a comprehensive and efficient automation script delivery process 
across various testing phases and sprints. 
Would you like more details on any specific phase or tool? ': 1 


< Update the following table summarizes the automation roadmap 
for the project > 
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Here’s an updated table summarizing the automation roadmap for your 


project: 
Objective 
Spri of Test — = Over Paniacnee 
nt Automatio Da all % oe ee 
= ate 
Set up the 
Initial 2024 automation 
Spri Setup and_ -08- 10% framework 
. os = O 
nt 1 Unit 15 ——- and develop 
Testing — initial unit 
test scripts. 
Develop 
and 
automate 
: : 2024. integration 
integration ec 
aes -08- 20% test scripts, 
——— =£sUng 30 automate 
data setup 
and 
teardown. 
Develop 
and 
automate 
oe Syehann 2024 system test 
a ae -09- 30% scripts, 
— ~2SUng 1s perform 
initial 
system 
testing. 
Spri Regression 2024 40% Identify 
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Objective 
of Test Targ Over 
Automatio all % 
n 


Comments 


critical test 
cases 


Testing -09- develop and 
30 automate 


regression 
test scripts. 


Develop 
and 


automate 


Performanc | 2024 performanc 


e Testin eit 30% e test 
———= scripts, 


integrate 
into CI/CD 
pipeline. 


S| 


Develop 
and 
automate 

; 2024 security 
Security. 10 60% test scripts, 


Testing 30 perform 


initial 
security 
testing. 


N 


User 024 70% Develop 
nt 7 Acceptance -11- and_ 
Testing A, automate 
(UAT) UAT 


) 
H. 


a" 
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Objective 
Spri of Test aa Over ne 
nt Automatio a all % ea eee 
- Date 
scripts, 
gather 
feedback 
from 
stakeholder 
s. 
Refine and 
optimize 
, Continuous 2024 pose SOESEAe. 
Spr Improveme -11- 80% based on_ 
nt 8 = 30. feedback 
= — and_ 
performanc 
e data. 
Ensure all 
Full tests are 
Spri Integration Ts integrated 
: at oa 90% and_ 
nt 9 and Final 15 perform 
Testing final 
testing. 
Spri Final 2024 100% Review all 
nt 10 Review and -12- test scripts, 
Documenta ol. finalize 
tion documentat 
ion, and 
prepare for 
deployment 
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Objective 
Spri of Test an Over ee 
nt Automatio —— all % ree 
— Date = 


This roadmap ensures a structured and phased approach to developing 
and integrating automation scripts across various testing phases, aligned 
with your project sprints. 

Would you like more details on any specific sprint or phase? 


5.4 Test Environment Strategy 


< Provide the details around the environments available for 
testing or pre-production. Brief about how code movement will 
happen from one environment to another post successful 
completion of the test phase > 


Answer:- Environments Available for Testing and Pre-Production 
1. Development Environment 
- Purpose: Used by developers to write and initially test their code. 
e« Activities: Unit testing, initial integration testing. 
« Tools: IDEs, local databases, version control systems. 
2. Test Environment 
« Purpose: Used by QA teams to perform various types of testing. 


« Activities: Functional testing, integration testing, regression 
testing. 


- Tools: Test automation tools (e.g., Selenium, JUnit), test 
management tools (e.g., Jira, TestRail). 


3. Staging Environment 
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- Purpose: A replica of the production environment used for final 
testing before deployment. 


e« Activities: User acceptance testing (UAT), performance testing, 
security testing. 


« Tools: Monitoring tools (e.g., Prometheus, Grafana erformance 


testing tools (e.g., JMeter, LoadRunner). 


4. Pre-Production Environment 


¢« Purpose: Used for final validation and testing before code is 
moved to production. 


- Activities: End-to-end testing, final regression testing, 
compliance checks. 


« Tools: Deployment tools (e.g., Jenkins, Ansible), security testin 


tools (e.g., OWASP ZAP). 


Code Movement from One Environment to Another 


1. Development to Test Environment: 


o Process: Once developers complete their code and initial 
unit tests, the code is committed to the version control 


svstem (e.gq., Git). 


o Automation: CI tools like Jenkins or GitHub Actions 
automatically build the code and run initial tests. 


o Deployment: If the build and tests pass, the code is deployed 
to the test environment for further testing’. 


2. Test to Staging Environment: 


o Process: After successful testing in the test environment, the 
code is tagged and merged into the staging branch. 


o Automation: CI/CD tools trigger automated builds and 
deployments to the staging environment. 
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o Deployment: The code undergoes UAT, performance, and 
security testing in the staging environment’. 


3. Staging to Pre-Production Environment: 


o Process: Once the code passes all tests in the staging 
environment, it is prepared for pre-production deployment. 


o Automation: Deployment scripts and tools ensure the code is 
deployed to the pre-production environment. 


o Deployment: Final validation and compliance checks are 
performed in the pre-production environment’. 


4. Pre-Production to Production Environment: 


o Process: After successful validation in the pre-production 
environment, the code is approved for production 
deployment. 


o Automation: CI/CD pipelines automate the deployment 
process to the production environment. 


o Deployment: The code is deployed to the production 
environment, and monitoring tools ensure it runs smoothly’. 


By following this structured approach and using appropriate tools, 
you Can ensure a smooth and efficient code movement process across 


different environments, leading to reliable and high-quality software 
releases. 


Would you like more details on any specific environment or 
deployment process? !: 5 


5.4.1 Test Environment / Pre-production Environment 


< Brief about the test environment details. The user access, rights 
and limitation of the environment. Provide which checks will be 
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carried post deployment of the code to ensure health of the 
environment is okay to move testing forward > 


Answer:- Test Environment Details 


1. Development Environment 
- Purpose: Used by developers for writing and initially testing their code. 
«- User Access: Developers have full access to modify and test their code. 


« Rights and Limitations: Full administrative rights, but changes here 
do not affect other environments. 


2. Test Environment 


- Purpose: Used by QA teams to perform various types of testing. 
« User Access: QA engineers and testers have access to execute tests 
and report issues. 
« Rights and Limitations: Limited to testing activities; no code 
modifications allowed. Access to test data and test scripts. 
3. Staging Environment 
- Purpose: A replica of the production environment for final testing 
before deployment. 
« User Access: QA engineers, testers, and sometimes product owners for 
UAT. 
« Rights and Limitations: Limited to testing and validation; no code 
changes. Access to staging data and configurations. 
4. Pre-Production Environment 
- Purpose: Used for final validation and testing before code is moved to 
production. 
- User Access: OA engineers, testers, and release managers. 
« Rights and Limitations: Limited to final testing and validation; no 
code changes. Access to pre-production data and configurations. 


User Access, Rights, and Limitations 
1. User Access: 


o Developers: Full access in the development environment, limited 


access in test and staging environments. 

o QA Engineers: Access to test and staging environments for 
executing tests and reporting issues. 

o Product Owners: Access to staging and pre-production 
environments for UAT. 

o Release Managers: Access to all environments for overseeing 
deployments. 

2. Rights and Limitations: 
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o Development Environment: Full rights for code changes and 
testing. 

o Test Environment: Rights to execute tests and access test data; 
no code modifications. 

o Staging Environment: Rights to perform UAT and validate 
changes; no code modifications. 

o Pre-Production Environment: Rights to perform final validation 
and compliance checks; no code modifications. 

Post-Deployment Checks 


1. Smoke Testing: 
o Purpose: Verify that the major functionalities of the application 


are working correctly. 
o Activities: Run automated smoke tests to ensure basic 


functionality is intact. 
2. Health Checks: 
o Purpose: Ensure the environment is stable and all services are 


running. 

o Activities: Monitor system logs, check service statuses, and verify 
database connectivity. 

3. Performance Monitoring: 

o Purpose: Assess the application’s performance under the current 
load. 

o Activities: Use monitoring tools like Prometheus and Grafana to 
track performance metrics. 


4. Security Scans: 
o Purpose: Identify any security vulnerabilities introduced by the 


new code. 


o Activities: Run automated security scans using tools like OWASP 
ZAP or Burp Suite. 
9. Regression Testing: 
o Purpose: Ensure that new changes have not adversely affected 
existing functionality. 
o Activities: Execute automated regression test suites to validate 
the stability of the application. 
By following these steps and using appropriate tools, you can ensure that 
the test environment is healthy and ready for further testing, leading to 
reliable and high-quality software releases. 
Would you like more details on any specific environment or post- 
deployment check? !: 1 *: 2 3:5 
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5.4.2 Production Environment 


< Brief about the test environment details. The user access, rights 
and limitation of the environment. Provide which checks will be 
carried post deployment of the code to ensure health of the 
environment is okay> 

Answer:- Sure! Here’s a brief overview of the test environment details 


user access, rights and limitations, and the checks to be carried out post- 
deployment: 

Test Environment Details 

A test environment is a dedicated setup where software testing activities 
are conducted. It replicates the production environment to ensure 
accurate and reliable testing. This environment includes hardware, 
software, network configurations, databases, and test data’. 


User Access, Rights, and Limitations 
1. User Access: 
o Developers: Typically have access to deploy code, run tests, and 


debug issues. 

o Testers: Have access to execute test cases, report bugs, and verify 
fixes. 

o Administrators: Have full access to manage the environment, 
including setting up and maintaining configurations. 

2. Rights and Limitations: 

o Read/Write Access: Developers and testers usually have 
read/write access to specific directories and databases relevant to 
their tasks. 

o Restricted Access: Sensitive areas, such as security 
configurations and critical system files, are restricted to 
administrators only. 

o Role-Based Access Control (RBAC): Ensures that users have the 
minimum necessary access to perform their roles, reducing the risk 
of unauthorized changes’. 

Post-Deployment Checks 
After deploying code to the test environment, several checks are 
performed to ensure the environment’s health: 
1. Smoke Testing: A quick set of tests to verify that the major 
functionalities are working correctly. 
2. Regression Testing: Ensures that new changes haven’t adversely 
affected existing functionalities. 
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3. Performance Testing: Checks the system’s performance under various 
conditions to ensure it meets the required standards. 
4. Security Testing: Identifies vulnerabilities and ensures that security 
measures are effective. 
9. Environment Monitoring: Continuous monitoring of system resources 
(CPU, memory, disk usage) to detect any anomalies. 
6. Log Analysis: Reviewing logs for errors or warnings that might indicate 
issues with the deployment. 
These steps help maintain a stable and reliable test environment, 
ensuring that any issues are identified and addressed promptly. 
Is there anything specific you’d like to know more about? 


5.5 Test Execution and Test Automation Strategy 


< Provide the details of which critical functionalities will be 
executed and how various aspects will be taken care i.e. Defect 
management, updating test case results in test management tools 
> 
Answer:- Critical Functionalities to be Executed 
1. User Authentication and Authorization: 
Verify login, logout, and session management. 


o Ensure role-based access control (RBAC) is functioning correctly. 
2. Data Integrity and Transactions: 
o Test CRUD (Create, Read, Update, Delete) operations. 


Validate data consistency across different modules. 


3. API Endpoints: 
o Check the functionality of all exposed APIs. 


o Ensure proper error handling and response times. 


4. User Interface (UI) Functionality: 


o Validate navigation, form submissions, and user interactions. 
o Ensure responsiveness across different devices and browsers. 
5. Performance Metrics: 
o Measure load times, throughput, and resource utilization. 


o Conduct stress and load testing to identify bottlenecks. 
Defect Management 
1. Defect Logging: 
o Use tools like JIRA, Bugzilla, or Azure DevOps to log defects. 


o Include detailed steps to reproduce, screenshots, and severit 
levels. 
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2. Defect Triage: 
o Prioritize defects based on impact and urgency. 


Assign defects to the appropriate developers for resolution. 


3. Defect Tracking: 
o Monitor the status of defects from identification to closure. 
o Conduct regular defect review meetings to ensure timel 
resolution. 
Updating Test Case Results in Test Management Tools 
1. Test Execution: 
o Execute test cases using tools like TestRail, HP ALM, or Zephyr. 
o Record the results (Pass/Fail) for each test case. 
2. Result Documentation: 
o Attach evidence (screenshots, logs) to the test cases in the tool. 
o Provide comments or notes for any deviations or observations. 
3. Reporting: 
o Generate test execution reports to track progress and coverage. 
o Share reports with stakeholders to keep them informed about the 
testing status. 
Ensuring Comprehensive Coverage 
1. Test Case Review: 
o Regularly review and update test cases to cover new functionalities 


and edge cases. 
o Ensure test cases are aligned with the latest requirements and 
specifications. 
2. Continuous Integration (CI): 
o Integrate automated tests into the CI pipeline using tools like 
Jenkins, GitLab CI, or Azure Pipelines. 
o Ensure tests are executed with every code commit to catch issues 
early. 
3. Feedback Loop: 

o Maintain open communication between developers, testers, and 

stakeholders. 

o Use feedback to improve test cases, processes, and overall quality. 
These steps ensure that critical functionalities are thoroughly tested, 
defects are managed efficiently, and test results are accurately 
documented, leading to a robust and reliable software product. 

Is there anything specific you'd like to dive deeper into? 
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< Also brief about the test automation tool, status of the scripts 
will be updated and the critical functionalities to be prioritized 
during continuous Validation > 

Answer:- Test Automation Tool 


One of the popular test automation tools is Selenium. It’s widely used 
for automating web applications for testing purposes. Here are some key 
features: 
« Cross-Browser Testing: Supports multiple browsers like Chrome, 
Firefox, Safari, and Edge. 
- Language Support: Compatible with various programming languages 
such as Java, Python, C#, and Ruby. 
«- Integration: Easily integrates with CI/CD tools like Jenkins, GitLab CL 
and Azure Pipelines. 
« Flexibility: Allows for the creation of complex test scripts to handle 
various testing scenarios’. 
Updating the Status of Scripts 
1. Automated Execution: 


o Scripts are executed automatically as part of the CI/CD pipeline. 


o Results are captured and logged in real-time. 


2. Reporting: 
o Use tools like Allure or ExtentReports to generate detailed test 
reports. 


o Reports include pass/fail status, execution time, and error logs. 
3. Dashboard: 


o Implement a dashboard (e.g., using Grafana or Kibana) to visualize 
test results and trends. 

o Provides a quick overview of the test execution status and helps in 
identifying patterns. 

Critical Functionalities to Prioritize During Continuous Validation 
1. Core Business Logic: 

o Ensure that the main functionalities, such as payment processin 

or user registration, are always working correctly. 


2. API Endpoints: 
o Continuously validate the functionality and performance of critical 


APIs. 
o Check for response times, data integrity, and error handling. 
3. User Authentication: 
o Regularly test login, logout, and session management features. 
o Ensure security measures like multi-factor authentication are 


functioning properly. 
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4. Data Integrity: 
o Validate that data is correctly stored, retrieved, and updated 
across different modules. 
o Perform consistency checks to ensure data accuracy. 
5. Performance Metrics: 
o Monitor key performance indicators such as load times 
throughput, and resource utilization. 
o Conduct regular performance tests to identify and address 


bottlenecks. 


6. Security: 
o Continuously test for vulnerabilities and ensure that securit 


patches are applied promptly. 
Validate that access controls and data encryption are effective. 


By focusing on these critical functionalities and maintaining an updated 
status of test scripts, you can ensure a robust and reliable software 
product throughout the development lifecycle. 


Would you like more details on any specific aspect? ': Test Automation 
Tools 


5.6 Defect Management Strategy 


< Provide the details of how defects will be logged to defect 
management tool. Also brief how defect would be treated over a 
defect life cycle and communication mechanism to notify of in 
the case of defect status change > 
Answers:- Logging Defects in a Defect Management Tool 
1. Defect Identification: 
When a defect is identified, it is logged into a defect management 


tool like JIRA, Bugzilla, or Azure DevOps. 

o The defect report includes details such as defect ID, summary, 
description, steps to reproduce, expected and actual results, 
severity, priority, and attachments (e.g., screenshots, logs). 

2. Defect Logging Process: 

o Create a New Defect: Navigate to the defect management tool 
and select the option to create a new defect. 

o Fill in Details: Enter all relevant information, including the 
environment where the defect was found, the module or feature 


affected, and any additional notes. 
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o Assign the Defect: Assign the defect to the appropriate developer 
or team for resolution. 

o Save and Submit: Save the defect report, which will then be 
added to the defect tracking system. 


Defect Life Cycle 


1. New: 
The defect is logged and awaits review. 


o Initial triage is performed to validate the defect and determine its 


severity and priority. 


2. Assigned: 
The defect is assigned to a developer or team for investigation and 


resolution. 
o The assignee reviews the defect details and starts working on a fix. 
3. In Progress: 
o The developer works on fixing the defect. 


o The status is updated to reflect that the defect is being actively 
addressed. 
4. Fixed: 
o Once the defect is resolved, the status is updated to “Fixed.” 
o The fix is deployed to a test environment for verification. 
5. Retest: 
Testers verify the fix by retesting the defect. 


o Ifthe defect is resolved, it moves to the next stage; if not, it is 
reassigned to the developer. 
6. Closed: 
o Ifthe defect is successfully fixed and verified, it is marked as 
“Closed.” 
The defect is considered resolved and no further action is required. 


7. Reopened: 
o Ifthe defect reoccurs or the fix is not satisfactory, it is reopened 
and reassigned for further investigation. 


Communication Mechanism for Defect Status Changes 
1. Automated Notifications: 


o The defect management tool sends automated email notifications to 
relevant stakeholders whenever there is a status change. 
o Notifications include details about the defect, the new status, and 
any comments or updates. 
2. Dashboard and Reports: 
o Stakeholders can view real-time status updates through 


dashboards and reports generated by the defect management tool. 
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o These dashboards provide a comprehensive view of all defects, 
their statuses, and progress. 


3. Regular Meetings: 
o Conduct regular defect triage and review meetings to discuss the 


status of critical defects. 


o Ensure open communication between developers, testers, and 
project managers to address any issues promptly. 


4. Comments and Mentions: 
o Use the commenting feature in the defect management tool to add 


notes, updates, and mentions. 
o Tag relevant team members to ensure they are notified of 
important changes or actions required. 
By following these processes, you can ensure that defects are logged, 
tracked, and resolved efficiently, with clear communication and 
visibility throughout the defect life cycle. 
Is there anything specific you’d like to explore further? 


5.7. Setting up Automated Gating criteria 


< Provide the details of which all critical functionalities must be 
successfully tested in order to promote the code to the next level 
or next phase of the continuous validation pipeline. Brief which 
tool being used and how automated gating criteria would be set 
up > 


Answers:- Critical Functionalities for Code Promotion 


To ensure a smooth promotion of code to the next phase in the 
continuous validation pipeline, the following critical functionalities 
must be successfully tested: 
1. User Authentication and Authorization: 
o Verify login, logout, and session management. 
o Ensure role-based access control (RBAC) is functioning correctly. 


2. Core Business Logic: 
Validate key business processes and workflows. 


o Ensure data integrity and consistency across modules. 


3. API Endpoints: 
Test the functionalit erformance, and security of all exposed 


APIs. 
o Ensure proper error handling and response times. 


4. User Interface (UI) Functionality: 
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o Validate navigation, form submissions, and user interactions. 

o Ensure responsiveness across different devices and browsers. 
5. Performance Metrics: 

o Measure load times, throughput, and resource utilization. 


o Conduct stress and load testing to identify bottlenecks. 


6. Security: 
o Perform security testing to identify vulnerabilities. 


o Ensure compliance with security standards and best practices. 


Tool Being Used 
A popular tool for test automation in CI/CD pipelines is Selenium. It is 
widely used for automating web applications and supports multiple 
browsers and programming languages. 
Automated Gating Criteria Setup 
Automated gating criteria ensure that only code meeting predefined 
quality standards is promoted to the next phase. Here’s how to set it 
up: 
1. Define Gating Criteria: 

o Unit Tests: All unit tests must pass. 

o Integration Tests: Critical integration tests must pass. 

o End-to-End Tests: Key end-to-end tests must pass. 

o Performance Benchmarks: Performance tests must meet 


specified thresholds. 

o Security Checks: No critical security vulnerabilities should be 
present. 

2. Implement in CI/CD Pipeline: 

o UseaClI/CD tool like Jenkins, GitLab CI, or Azure Pipelines to 
automate the process. 

o Configure the pipeline to run the defined tests and checks at each 
stage. 


3. Automated Approval: 
o Set up automated gates that check the results of the tests. 


o If all criteria are met, the code is automatically promoted to the 
next phase. 

o If any criteria fail, the pipeline stops, and notifications are sent to 
the relevant stakeholders. 

4. Continuous Monitoring: 

o Use monitoring tools like Grafana or Kibana to visualize test 
results and trends. 

o Continuously monitor the health of the pipeline and make 


adjustments as needed. 
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By ensuring these critical functionalities are thoroughly tested and 
setting up automated gating criteria, you can maintain a high level of 
quality and reliability in your software development process’”?. 
Would you like more details on any specific aspect? ': Fast track code 
promotion in your CI/CD pipeline *: From Code to Production: A 
Comprehensive Practical Guide to CI/CD Pipelines *: Continuous 
Validation: What Is It And Why Is It Important? 


5.8 Reporting & Test Metrics 


Following key metrics will be captured as part of 
the metrics dashboard. 


Metric Definition Frequen 
cy 
Defect Defect Acceptance is % ratio of 
Acceptance Ratio | number of defects accepted to 
(DAR) the total number of defects 
raised. 


Defect Removal DRE is a ratio of defects raised 
Efficiency (DRE) | against the sum of post- 
production defects and total 
number of defects raised. 


% Deferred Deferred defects is a ratio of sum 

Defects of Open Defects and deferred 
defects against total number of 
defects 


Reopen defects Reopen defects is a count of 
defects reopened after retesting 
Test case % ratio of test cases executed vs. 
Execution (TCE) | test cases planned 

Test Automation | Total Test Case Actually 
Coverage Automated ( For 
Application)/Total Test Cases 
feasible to automate 

Execution cycle’ | Time taken to execute all the 
time in scripts in a particular test phase 


©2016 Infosys Limited, India 


Infosys Test Strategy for Continuous Validation 


continuous of continuous validation 
validation 


Reporting & Test Metrics 


Here are the key metrics that will be captured as part of the metrics 
dashboard: 


Metric Definition ereuuen 
——— ee cy 
Defect Acceptance is the 
Defect ; : 
rare : percentage ratio of the number of Continuo 
Acceptance Ratio def d _ 1 a 
DAR elects accepted to u e tota us 
(DAR) number of defects raised. 
DRE is the ratio of defects raised 
Defect Removal against the sum of post- Continuo 
Efficiency (DRE production defects and the total us 
number of defects raised. 
Deferred defects is the ratio of 
% Deferred the sum of open defects and Continuo 
Defects deferred defects against the total us 
number of defects. 
Peanen Dele Reopen defects is the count of Continuo 
neopen Verects 
defects reopened after retesting. us 
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Metric Definition Frequen 
cy 
Test Case The percentage ratio of test cases Continuo 
Execution (TCE) executed vs. test cases planned. us 
Total test cases actually 
Test Automation automated (for the application) Continuo 
Coverage divided by the total test cases us 
feasible to automate. 
i . 
Execution Cycle_ ion Cycle Time taken to execute all the : 
Time in aco cr the GaSN ET SIeT aor ane ee Continuo 
eee scripts in a particular test phase <r 
Continuous ; ; ; us 
: ; of continuous validation. 
Validation ieee acer eens 


These metrics provide valuable insights into the quality and efficiency of 
the testing process, helping to identify areas for improvement and ensuring 
that the software meets the required standards before moving to the next 
phase. 


Would you like more details on any specific metric or how to implement 
these in your dashboard? 
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6. Reference(s) 


if P ontnnieus Validation SharePoint: 


ee: ee aspx 
Overall DevOps Strategy: Link to be updated 


User Story document: Link to be updated 
Gap Analysis: Link to be updated 
Application User Stories: Link to be updated 
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